Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hotfix/update2new dsx tone analyzer #5

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

tjhgit
Copy link

@tjhgit tjhgit commented Mar 16, 2018

DSX now runs on version 2.11.8 and scala 2.1.2. Also tone analyzer changed the return value since 2017.

  1. use new versions in build.sbt
  2. got rid of Logging -> only using log4j
  3. JSONObject is now provided by different package

Tested this pull request as follows:

To compile it, use this command: docker run -it -v $(pwd):/root/src -p 4040:4040 -p 8080:8080 -p 8081:8081 -h spark --name=spark p7hb/docker-spark:2.1.1 when inside the above spark.samples directory.

Inside the docker container:

cd /src/streaming-twitter

sbt update
sbt clean compile
sbt console

sbt assembly

Runs flawlessly in sbt console, however leads to the following error on DSX:

Name: Compile Error
Message: :42: error: type mismatch;
found : org.apache.spark.org.apache.spark.org.apache.spark.org.apache.spark.org.apache.spark.SparkContext
required: org.apache.spark.org.apache.spark.org.apache.spark.org.apache.spark.org.apache.spark.SparkContext
demo.startTwitterStreaming(sc, Seconds(30))
^
StackTrace:

… current dsx scala and spark version, modified old Logging class to use only log4j, JSONObject now in a different package
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants